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Abstract 

This study explored the effectiveness of using Blackboard, an online software program, to 
enhance a graduate course in school counseling. Research questions were: (1) Does Blackboard 
enhancement facilitate student learning?; (2) Is there a difference in student response to 
Blackboard by class site (i.e., off-campus v on-campus)? Materials made available through 
Blackboard were: syllabus, counselor standards, professional information, lecture notes, and 
practice tests. Effectiveness measures were usage records from Blackboard, end-of-course 
student evaluations, and instructor's evaluation. Students in both sections agreed that Blackboard 
enhancement increased their learning (M=6.68, SD=0.48; scale range l=Strongly disagree to 
7=Strongly agree). There was no difference in students' rating of satisfaction with Blackboard by 
class site. 
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Evaluation of Electronic Blackboard Enhancement of a Graduate Course in School Counseling 

The purpose of this exploratory descriptive research study, funded by Preparing Teachers 
to Teach Technology (PT 3 ) Grant money, was to examine the effectiveness of using Blackboard, 
an online software program, to enhance the teaching of a graduate course in school counseling. 
Two research questions drove the inquiry: (1) Does Blackboard enhancement facilitate student 
learning? and (2) Is there a difference in student response to Blackboard enhancement by class 
site, that is, off-campus versus on-campus? 

Theoretical Framework 

Blackboard is one of the more popular software tools provided to university faculties to 
for online and web-enhanced teaching. Crawford and Thomas-Maddox (2000) and Ruman and 
Gillette (2001) presented a excellent in-depth descriptions of Blackboard as well as practical 
ideas for using the software. Velayo (2001) reported evidence of effective pedagogy for both 
completely online and Blackboard enhanced psychology courses. There is a rapidly growing 
body of research examining the effectiveness of online teaching (e.g., Howell, 2000; Jansak, 
2000; Nicoll & Laudato, 1999; Phipps & Merisotis 2000). However, an ERIC search revealed no 
research that explored the online teaching of counseling courses. 

Undoubtedly, there are elements of counselor education, such as developing counseling 
techniques, that require the face-to-face interaction of a traditional classroom setting. Further, 
there is much incidental learning that occurs among students and between students and instructor 
in face-to-face classrooms. The counselor education faculty at the regional state university in 
which this research was situated was resistant to placing counseling courses completely online. 
Thus, this study was undertaken to explore the effectiveness of using Blackboard to enhance, 
rather than to replace, a face-to-face school counseling graduate-level course. 
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Method 

Participants 

The participants in this stijidy were 32 graduate students enrolled in a required school 
counseling course in the Masters in Counseling program. Ten students (nine female and one 
male) were enrolled in the off-canjipus section held Tuesday evenings in a small town 75 miles 
from the main campus. Twenty-two students (18 female and four male) were in enrolled in the 
on-campus section held Wednesday evenings in a classroom in the College of Education 
building on the main campus of th<f regional state university. Thirty-one of the students were 
currently employed as public school teachers. Only one student, an international student from 
Turkey, was a full-time graduate situdent. Only five of the students had previously taken a 
Blackboard enhanced class. All of die students were proficient in the use of e-mail and had ready 
access to the Internet. 

Classroom Sites 

The classrooms for the face-t.o-face classes were very different. The on-campus 
classroom in was well lighted, well ventilated and was equipped with chalkboard, chalk, 
overhead projector, and a television/'VCR. The off-campus class was 75 miles from the main 
campus and had been scheduled to me:et at the university's center in a small rural community. 
However, due to unexpectedly high enrollments in other classes, the school counseling class was 
moved out of the university's center to the second floor library of a Methodist Church. The 
library was a small room with no chalkboard, no overhead projector, and no television/VCR. 

Two library tables with chairs provided adequate seating and the room was well-lighted and 
ventilated. The small number of students (1_0) ai.d the setting provided a "cozy" atmosphere for 
the counselor education class. 
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Procedure 

The proposal for this study was reviewed and approved by the Human Subjects Review 
Committee at the university. With the agreement that no data would be examined until the final 
grades had been submitted to the registrar, the Committee did not require that the students be 
informed of the study. 

The instructor e-mailed directions for enrolling in the Blackboard sites when the students 
enrolled in the two sections of the class. The university provided separate Blackboard sites for 
each section of the course. As students entered Blackboard, announcements greeted them and 
directed them to the course syllabus and the state's New School Counselor Standards. (Students 
used the seven New Counselor Standards to organize a professional portfolio-a semester-long 
project.) New Blackboard materials were posted each Thursday before 3:00 p.m. Materials 
placed on the (identical) Blackboard sites were: (1) the course syllabus, (2) links to the state 
department of education's website, (3) links to professional organizations, for example, 

American Counseling Association, (4) links to related educational materials, for example, Buros 
Mental Measurement Yearbook site, (5) weekly lecture notes, (6) short practice tests to reinforce 
lectures, (7) mid-term and final exams (both were take-home exams). (An annotated list of the 
online resources used to enhance this course can be found in the Appendix of this paper.) 

Data Sources 

Effectiveness measures included (1) usage-tracking records from Blackboard, (2) end-of- 
course student evaluations, and (3) instructor's evaluation. Blackboard software provides a 
"tracking" option that allowed the instructor to examine the number of times that each student 
visited specific materials placed on Blackboard. Additionally, the instructor was able to track the 
time of day that the student visited the materials. The student evaluations of Blackboard were 
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collected during the last class meeting, after the take-home final exam had been collected. The 
evaluation sheets were placed in a sealed envelope and were examined after the students' final 
grades were posted with the registrar's office. 

Results 

Blackboard Usage 

The frequency of student visits to specific Blackboard areas are presented in Table 1. The 
percentage refers to percentage of Blackboard areas visited, not the percentage of students. 



Table 1. Frequency and Percent of Student Visits to Blackboard Areas 



Off-Campus Students (N= 


10) 


On-Campus Students (N=22) 


Blackboard Area 


Frequency of 


Percent 


Blackboard Area 


Frequency 


Percent 




Visits 


of Visits 




of Visits 


of Visits 


Communication Areas 


278 


9.57% 


Communications Areas 


208 


5.89% 


Main Content Areas 


2435 


83.84% 


Main Content Areas 


3050 


86.42% 


Group Areas 


9 


0.30% 


Group Areas 


14 


0.39% 


Student Areas 


182 


6.26% 


Student Areas 


257 


7.28% 


Total 


2904 


100% 


Total 


3529 


100% 



The Blackboard areas visited by off-campus and on-campus students were very similar with the 
exception of the Communications Areas (Table 1). The percentage of visits for the off-campus 
students (9.57%) was greater than the percentage (5.89%) visits by the on-campus students. 

Because the number of students in each class was different, a mean "frequency of use" 
score was calculated for each class by dividing the total number of Blackboard visits by the 
number of students in the class. Thus, an average number of visits per student was calculated as a 
"frequency of use measure." For the Off-Campus class, the frequency of use was 290.4 (average 
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number of visits per student). For the On-Campus class, the frequency of use was 160.4. This 
statistic is somewhat problematic because one very interested student could skew this mean 
score. However, it allows for a rough estimate of the differences in visits by class section. 

Student Evaluation Ratings 

The instructor developed a Student Evaluation Rating instrument consisting of 22 items 
followed by seven-point Likert-type item rating scales (l=strongly disagree to 7=strongly agree). 
Table 2 presents a summary of the data collected with this instrument by class site. 
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Table 2. Means of Student Evaluation Ratings by Class Site 



Item 


Off-Campus 

(N=10) 


On-Campus 

(N=22) 


M 


SD 


M 


SD 


1 . The syllabus of this course clearly explained the course requirements 


6.80 


0.42 


6.55 


0.74 


2. 1 had easy access to a computer with Internet access. 


6.40 


1.26 


6.86 


0.35 


3. I used Blackboard to review the course syllabus. 


5.70 


1.57 


6.55 


0.74 


4. The directions provided for accessing Blackboard were clear and accurate. 


6.90 


0.32 


6.32 


0.99 


5. 1 used Blackboard to access lecture notes for the class. 


7.00 


0.00 


6.59 


1.30 


6. I used Blackboard to communicate with the instructor of this class. 


4.50 


2.12 


5.64 


1.71 


7. I used Blackboard to communicate with fellow students in this class. 


4.00 


2.11 


3.23 


2.27 


8. The "links" on Blackboard to supplemental material (e.g. the Research Page) 
increased my learning in the course. 


6.30 


1.06 


6.05 


1.36 


9. 1 used the "practice quizzes" on Blackboard. 


6.40 


0.84 


5.91 


1.66 


10. The "practice quizzes" on Blackboard increased my learning. 


5.90 


1.10 


5.55 


1.63 


11. The class that was conducted entirely through Blackboard was as 
effective as the face-to-face sessions of the class. 


4.90 


2.08 


6.00 


1.54 


12. 1 was able to access, download, and print material from the Internet for my 

professional portfolio. 


6.70 


0.95 


6.77 


0.43 


13. 1 had used the Internet for previous courses. 


5.40 


2.59 


6.09 


1.82 


14. Overall, web-enhancement increased my learning. 


6.70 


0.67 


6.68 


0.48 


15. I would look forward to taking another web-enhanced course. 


6.50 


0.85 


6.73 


0.55 


16. I liked the practice quizzes the best. 


5.50 


1.35 


3.91 


1.80 


17. The links to other sites were the most useful aspect of Blackboard. 


5.50 


1.84 


5.09 


1.72 


18. 1 used the Blackboard site to send an E-mail to groups (or all) of the 
students in the class. 


3.70 


2.50 


3.09 


2.35 


19. I would like to use Blackboard to enhance my teaching (or work). 


6.10 


1.20 


5.36 


1.53 


20. My typing skills are excellent. 


5.10 


1.10 


5.59 


1.99 


21.1 would have liked it if this class had been entirely on line. 


4.80 


2.20 


5.77 


2.02 


22. 1 have taken a course that was taught entirely on line. 


Yes = 4 40% 

No = 6 60% 


Yes = 13 
59% 

No = 9 41% 



Note: Scale range l=strongly disagree to 7=strongly agree 
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With two exceptions, all items were rated above the scale's neutral midpoint of four, Item 
7, "I used Blackboard to communicate with fellow students in this class," and Item 18, "I used 
the Blackboard site to send an E-mail to groups (or all) of the students in the class." Clearly, the 
Discussion Board and group E-mail functions of Blackboard were not widely used. The 
relatively large standard deviations for these items indicated that there was greater variation in 
students' reported use. 

The high mean ratings for Item. 12, "I was able to access, download, and print material 
from the Internet for my professional portfolio," was important because this was one of the main 
uses of Blackboard planned by the instructor. The high ratings of Item 15, "I would look forward 
to taking another web-enhanced course," was very encouraging to the instructor. An invitation 
for additional comments on the evaluation rating instrument drew responses from only nine of 
the 32 students. The comments were all positive and can be characterized by, "Enjoyed the 
class." Two students (on campus) suggested that the course would have been better if it had been 
entirely online; one student (off campus) stated, "Blackboard/Intemet is great, it adds to the 
class, but face-to-face instruction is a must." 

Although the small size (N=10) of the class meeting in the library of a Methodist Church 
off-campus permitted a less formal presentation of material, having Blackboard available for 
extending learning was extremely valuable for the instructor. In both classes, the students printed 
the lecture notes from Blackboard and brought them to class (Note that Item 5 evaluating this 
practice was rated 7.00 on the 7-point scale by the Off-Campus class.) This allowed for more 
student discussion of issues presented through lecture (less time note taking) and the face-to-face 
classes became more interactive and personal. 



Evaluation of Electronic Blackboard 10 



With one exception, all of the students were teaching in the public schools during the day 
and they were able to share more of their real-world experiences that illuminated the concepts 
presented through lecture. A great deal of "incidental" learning took place through the face-to- 
face interactions. 
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Differences in Ratings by Students by Class Site 

Chi-Square Tests for Independence (Gravetter & Wallnau, 1996) were used to test for 
differences in students' mean ratings of items by class site. Tested with alpha set at .05, no 
statistically significant differences were found. 

Student Usage by Day of the Week 

Tables 3 and 4 present data describing student usage by day of the week. Table 3 presents 
data from the Off-Campus class. The class met Tuesday evenings from 6:00 - 8:40 p.m. 

Table 3. Off-Campus Students' Visits bv Dav of the Week 



I000-T 

900 - 

800 - 

700 - 

600 - 

500 - 

400 - 

300 - 

200 - 

100 - 

0 - 



El Sunday □ Monday B Tuesday □ Wednesday □ Thursday □ Friday 11 Saturday 
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Table 4 presents On-Campus student usage of Blackboard by day of the week. This class 



met every Wednesday evening from 6:00 to 8:40 p.m. 



Table 4. On-Camnus Students' Usage bv Day of the Week 




□ Sunday □ Monday BTuesday □ Wednesday 

□ Thursday □ Friday 0 Saturday 



Students in the Off-Campus class accessed Blackboard most frequently on Monday and Tuesday 



(Tuesday evening class); Students in the On-Campus class accessed Blackboard most frequently 



on Wednesday (Wednesday evening class). 



Table 5. Off-Camous Students' Usage bv Hour of the Day 



400 

300 

200 

100 

0 







□ 10 a.m. 


■ 1 1 a.m. 


□ 12 noon □ 1 p.m. 


■ 2 p.m. 


□ 3 p.m. 


■ 4 p.m. 


□ 5 p.m. 


■ 6 p.m. 


□ 7 p.m. D8p.m. 


□ 9 p.m. 


■ 10 p.m. 


■ 11 p.m. 



Note: All times not charted had zero usage. 
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The data presented in Table 5 show that the students in the Off-Campus class most 
frequently accessed Blackboard at 1 1:00 a.m. The would indicate that students accessed 
Blackboard at school (most frequent days were Monday and Tuesdays). Table 6 presents On- 
Campus student usage by time of day. 



Table 6. On-Campus Students' Usage bv Time of Dav 




□ 10 a.m. 


■ 1 1 a.m. 


11 12 noon 


i □ 1 p.m. 


■ 2 p.m. 


□ 3 p.m. 


□4 p.m. 


□ 5 p.m. 


■ 6 p.m. 


□ 7 p.m. 


□ 8 p.m. 


□ 9 p.m. 


■ 10 p.m. 


■ 11 p.m. 





Note : All times not charted had zero usage. 

The On-Campus students most frequently accessed Blackboard at 2 p.m. followed by 3 
p.m. On-Campus access was most frequent on Wednesday, the day of class. Thus, teachers most 
frequently accessed Blackboard at school before coming to class on Wednesday evening. 
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Conclusions 

This study adds to the very scant literature on using online enhancement for counselor 
education courses. Blackboard enhancement was very well received by the students in both the 
off-campus and on-campus sections of a graduate school-counseling course. There was no 
statistically significant (p.<.05) difference in the mean ratings on the students' evaluation forms 
by class site (on-campus versus off-campus). 

Frequency of use was difficult to measure because the calculation of a "frequency of use" 
mean by dividing the number of student visits by the number of students in each class could 
easily be skewed by very frequent use by a single student (as was the case in the on-campus site). 
Students (all but one employed as full-time classroom teachers) most frequently visited the 
Blackboard site the day before or the day of the evening class. The students most frequently 
accessed Blackboard at their schools. 

From the instructors' point of view, Blackboard was most useful to facilitate learning by 
providing student access to materials outside of class time. The class was organized around 
Kentucky's New Counselor Standards (seven). Using Blackboard to guide students to external 
links that contained information for their portfolios (see Appendix) was an extremely productive 
use of this technology. Blackboard enhancement provided more face-to-face class time for 
student discussion and interaction— an important element in a counseling course. 
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APPENDIX 

Annotated List of Websites 
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American Counseling Association Code of Ethics 

(http://www.counseling.org/resources/codeofethics.htm) 

American Counseling Association Testing Ethics 

(http://www.counseling.Org/resources/codeofethics.htm#ce) 

The ACA Ethics of Testing 

American School Counselor Association (http://www.schoolcounselor.org/) 

ASCA with lots of information and links for school counselors. 

Brain-Based Learning and Assessment 

(http://www.brainconnection.com/library/?main=eduhome/assessment-intelligence) 

This site has links to research studies on "brain-based" learning and assessment. 

Buros Mental Measurement Yearbook (http://www.unl.edu/buros/) 

Explore the buttons on this link. Search for a test by subject. 

Career Clusters (http://www.kde.state.ky.us/osis/voced/proginfo.asp) 

Have you used "Career Clusters" for vocational guidance? Are they used in your school? 
How might these work in elementary school? 

Carl Rogers (http://www.ship.edu/~cgboeree/rogers.html) Information about the father of 
Client-Centered Therapy. We should hold him in unconditional positive regard perhaps. 
This site offers good descriptions of Carl Rogers' philosophy and practice. 

CATS Matrix - What test is given in which grades? 

(http://www.kde.state.ky.us/comm/commrel/cats/matrix.htm) 

You probably already know this, but here is a quick overview. 

Elementary School Counselor's Website (http://www.falmouthschools.org/k2counseling/) 
This is one excellent example of an elementary school counseling website. 




19 



Evaluation of Electronic Blackboard 19 



! 



High School Counselor's Link (http://www3.dist214.kl2.il.us/guidance/) 

This is a list of resources for high school counselors— contains a lot of information 
and links to other useful websites. Portfolios in the school counselor course are 
differentiated by school level. That is, students complete an Elementary Portfolio, 
Middle-School Portfolio, or High School Portfolio. 

Kentucky's Core Content for Assessment 

(http://www.kde.state.ky.us/oapd/curric/corecontent/core_content_index_version_30.asp) 

Although this is state specific, many state Department of Education websites include 
the content for the statewide accountability test. The student in the school 
counseling course are asked to link the New Counselor Standards with the Core 
Content (statewide curriculum guide). 

Kentucky's New Counselor Standards (http://www.kde.state.ky.us/osle/coun_stan.pdJ) 

This website lists Kentucky's New Counselor Standards (7). These were used as 
the organizing structure for the school counseling course. Readings and lectures 
were organized around the seven content areas of the standards. 

Kentucky Department Education 's Scoring Guide for Parents 

(http://www.kde.state.ky.us/comm/commrel/cats/scoring_guide.asp) 

Check out the guide for both Kentucky Core Content Tests (KCCT) and CTBS. 

This site offers scoring information on the statewide accountability test. 

Kentucky's Student Performance Standards 
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( 

(http://www.kde.state.ky.us/comm/pubinfo/standards/) 

Choose the standard for the grade level content area in which you teach (or closest grade 
to yours). Look at the definition of "Proficient" only. What do you think? 

Kiersey Personality Test (http://www.keirsey.com/) 

Take a personality test online. How is this similar to the MBTI? What is the theoretical 
base? 

Links to Counseling Resources on the NET (http://www.csun.edu/~hfedp001/edpsych.html) 

This is a comprehensive list of links by counseling categories. 

Mensa site (Check your IQ?) (http://www.mensa.org/) 

This site describes Mensa and offers a short IQ "workout". Perhaps an intellectually 
gifted middle school or high school student would be interested in 
this site? 

National Standards for School Counseling 

(http://www.cde.ca.gov/spbranch/ssp/natlstandards.htm) 

After extensive research and review, the American 
School Counselor Association announces the establishment 
of The National Standards for School Counseling Programs. 

Redefinition of School Counselor's Role 

(http://www.coe.ohio-state.edu/paes/DeWitt-Wallace/Definition.htm) 

This site offers information on the DeWitt- Wallace Program's Redefinition of School 
Counseling. 

Research on School Counselor Effectiveness 
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« 

% 

« 

(http://www.cde.ca.gov/spbranch/ssp/couneffct.pdf) 

School Counselor Position Statements 

(http ://www. schoolcounselor. org/content.cfrn?L 1=1 &L2=8) 

This will take you right to the Position Statements. (You need to choose 5 for portfolio). 
Test Released Items (with answers) from Kentucky's CATS 

(http://www.kde.state.ky.us/oaa/implement/KCCT_release/KCCT_items.asp) 
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